Section: New Results

Marked point processes

Multiple target tracking using spatio-temporal marked point processes

Participants : Paula Craciun, Josiane Zerubia [contact] .

This work has been done in collaboration with Mathias Ortner from Airbus D&S (http://www.space-airbusds.com/fr/ )

Multiple target tracking, stochastic geometry, point processes, remote sensing

Tracking can be defined as the problem of estimating the trajectories of objects in the image plane, as they move around the scene. Hence, a tracker assigns consistent labels to the objects in different frames of a sequence of images and can additionally provide information about the orientation, shape or size of the objects. Multi-target tracking has been historically achieved using sequential techniques, the major drawback of such methods residing in the impossibility to modify past results in the light of new data. However, applications such as offline video processing or information retrieval are not sequential in nature. Batch processing methods are preferred in this case since they do not suffer from the limitations of sequential methods. Nevertheless, these techniques remain poorly explored and highly underused.

We propose a novel approach based on spatio-temporal marked point processes to detect and track moving objects in a batch of high resolution images  [17] . We develop a new, intuitive energy based model consisting of several terms that take into account both the image evidence and physical constraints such as target dynamics, track persistence and mutual exclusion. We construct a suitable optimization scheme that allows us to find strong local minima of the proposed highly non-convex energy  [9] . The model has been validated on two types of data: remotely sensed satellite image sequences, characterized by high resolution, high signal to noise ratio and low temporal frequency; and biological image sequences, characterized by high resolution, low signal to noise ratio and high temporal frequency.

Tracking results are shown in Figure 4 , which shows the detection (dots) and tracking (lines) results of boats in a sequence of 14 high resolution remotely sensed images. The images are captured with a low temporal frequency at different acquisition angles.

Initialization and estimation of parameters for marked point processes applied to automatic object detection on satellite images

Participants : Aurélie Boisbunon, Josiane Zerubia [contact] .

This work has been done in collaboration with Rémi Flamary (Université de Nice Sophia Antipolis), Alain Rakotomamonjy (Université de Rouen) et Alain Giros (CNES). It was partially funded by the French Spatial Agency CNES [http://www.cnes.fr ].

Sparse representations, large scale, stochastic algorithms, machine learning, image processing

Marked point processes (MPP) strongly rely on parameters, whose estimation affects both computation time and performances. In this work, we proposed two approaches: the first one consists in initializing MPPs with a first coarse solution obtained very quickly from sparse regularization methods [7] , while the second one estimates the parameters by the Stochastic Approximation Expectation-Maximization (SAEM) algorithm [8] . We give details on both approaches below.

The first coarse solution is obtained from a deterministic sparse regularization method. This method is based on the representation of an image with objects as a sum of convolutions between atoms of a dictionary and matrices of positions of the objects in the image. Such a representation is displayed on Figure 5 . The atoms of the dictionary are fixed in advance and correspond to different instances of the objects (scales, angles, shapes, etc). This way, we transform the problem of object detection into the problem of estimating extremely sparse matrices. The algorithm we derived for solving the associated optimization problem is both parallelized and very efficient.

Up to recently, the parameters of MPPs were estimated by the Stochastic Expectation-Maximization (SEM) algorithm developed by Celeux & Diebolt (1985). This algorithm consists in alternatively estimating the expected pseudo-likelihood based on a random configuration and updating the parameter value by maximum of pseudo-likelihood. However, since it does not have a pointwise convergence, Ben Hadj et al. (2010) considered running a simulated annealing scheme after few iterations of SEM in order to reach convergence, at the cost of a higher computational time. Instead, we proposed to adapt the Stochastic Approximation Expectation-Maximization (SAEM) algorithm, developed by Delyon et al. (1999), to MPPs. Indeed, it both offers pointwise convergence and a similar computational time as SEM by efficiently taking into account past configurations in the update of the expected pseudo-likelihood.

Using both approaches resulted in the division of the computational time of the estimation of MPPs parameters by 2 and in an increase in performance of detection.

Generic curvilinear structure modeling via marked point process theory

Participants : Seong-Gyun Jeong, Yuliya Tarabalka, Josiane Zerubia [contact] .

Curvilinear structure extraction, object detection, marked point process, stochastic inference

We proposed a marked point process model to analyze underlying curvilinear structure for wide ranges of input data, for instance, wrinkles, DNA filaments, road cracks, and blood vessels [12] , [13] . It is based on sampling technique so that the model represents an arbitrary shape of the line network with a set of small line segments. The line segments should be fit into the given image data, and be harmonic with those of neighborhoods. To take these issues into consideration, we formulate a maximum a posteriori (MAP) estimation as an energy minimization problem. The energy function for given line configuration $𝐬$ can be decomposed into data likelihood term $E_{\mathrm{data}}$ and prior term $E_{\mathrm{prior}}$:

where $\#\left(𝐬\right)$ denotes the total number of line segments in the current configuration, $i\sim j$ represents symmetric neighborhood system, and $\lambda$ controls the relative importance of two terms. For the data term, we exploit oriented gradient information and homogeneity of the pixel intensities corresponding to line segment on the image site. The prior energy defines topology of the line configuration in that penalizes overlapping and attracts smooth connections. Another contribution of the work is to reduce parameter dependencies of the marked point process model using aggregation approach. We repeated to perform Markov chain Monte Carlo (MCMC) sampling with different parameter vectors to obtain multiple line hypotheses. Then, we combine line hypotheses to maximize the consensus among detection results.

In figure 6 , we have compared line detection results of manually labeled image, morphological filtering (path opening), supervised feature learning, an MPP model using single parameter vector, and the proposed algorithm. The proposed algorithm extracts the most salient line structures for all datasets without any parameter estimation procedure.